feat(ai-openrouter): forward system-prompt cache_control breakpoints#777
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds per-system-prompt ChangesOpenRouter system prompt cache_control support
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The text adapter collapsed `systemPrompts` to a plain joined string and dropped the object-form `metadata`, so Anthropic-family prompt caching over OpenRouter was unreachable — `cache_control` never reached the wire. Declare `OpenRouterSystemPromptMetadata` (threaded through `BaseTextAdapter`'s `TSystemPromptMetadata`, so `cache_control` is typed/autocompleted at the `chat()` call site) and, when any system prompt carries `cache_control`, emit the system message as a content-array part carrying the directive — mirroring `@tanstack/ai-anthropic`. Without `cache_control` the system message is still the same joined string, so existing callers are unaffected.
e121468 to
6eccb78
Compare
|
View your CI Pipeline Execution ↗ for commit 0b7f36a
☁️ Nx Cloud last updated this comment at |
@tombeckenham thank you! (and the rest of the team 😄) |
🎯 Changes
@tanstack/ai-openrouter's text adapter collapsedsystemPromptsto a plain joined string and dropped the object-formmetadata, so Anthropic-family prompt caching over OpenRouter was unreachable — acache_controlbreakpoint never reached the wire. (OpenAI routes auto-cache prefixes, so only Anthropic-family routes were affected.)This mirrors what
@tanstack/ai-anthropicalready does:OpenRouterSystemPromptMetadata({ cache_control?: ChatContentCacheControl }) and thread it throughBaseTextAdapter'sTSystemPromptMetadataparam, socache_controlis typed and autocompleted at thechat()call site.mapOptionsToRequest, when any system prompt carriescache_control, emit the system message as aChatContentText[]content array withcacheControlon the part(s) that declared it. Withoutcache_control, the system message is sent as the same joined string as before — existing callers are unaffected.Tests cover: a single cached prompt (asserting both the camelCase pre-serialization shape and the snake_case
cache_controlwire output viaChatRequest$outboundSchema), mixed cached/uncached prompts (breakpoint only on the prompt that declared it), and the unchanged no-cache_controlpath (foreign metadata still dropped, plain string preserved).✅ Checklist
pnpm run test:pr.🚀 Release Impact
Summary by CodeRabbit
cache_controldirective, which is forwarded to OpenRouter on supported routes.systemPromptsentries that carry cache control.cache_control.